ED 134 148 



DOCUHBHT BESUHE 

IB 004 307 



AOTHOB 
TITLE 

INSTITOTION 

PDB DATE 
IJOTE 

EDBS PBICE 
DESCBIPTOBS 



Nelson, Carol B, 

Computer-Produced Bibliographies for Class Use*. 
California Univ,, Los Angeles, Graduate School of 
Library and Information Science, 
Dec 76 

98p,; For related document, see ED 121 242 

HF-$0,83 HC-$U.67 Plus Postage, 
♦ Bibliographies; ♦Co'aputer Oriented Programs; 
Information Sources; Iiiformation Systems; Library 
Automation; ^Library aeterence Services 



ABSTBACT 

This paper presents data and discusses: (1) the 
procedures used in the preparation and delivery of eight 
computer-produced bibliographies, (2) the benefits of using them, (3) 
the costs in dollars, librarians, and time, (4) the possible 
operational problems, (5) suggested improvements, (6) additional 
uses, and (7) questions concerning service continuation. The data 
collection instruments vere guestionnedres, interviews, and 
"operational procedures recording forms," The main long term benefit 
was positive library experience. Short term benefits included: (1) 
time saved in locating references, and (2) a place to begin for 
students searching a new topic, (Author/DAG) 



« Documents acquired by EBIC include many informal unpublished * 

* materials not available from other sources, EBIC makes every effort * 

* to obtain the best copy available. Nevertheless, items of marginal * 

* reproducibility are often encountered and this affects the quality * 

* of the microfiche and hardcopy reproductions EBIC makes available * 

* via the EBIC Document Beproduction Service (EDBS), EDBS is not * 

* responsible for the quality of the original document. Be productions * 

* supplied by EDBS are the best that can be made from the original, * 



CGMPUTER. PRODUCED 
BIBLIOGRAPHIES FOR CLASS USE 



CAROL R. NELSON 



December 1976 



-PERMISSION TO REPRODUCE THIS COPY- 
RIGHTED MATERIAL HAS BEEN GRANTED BY 

Carpl__R. Nelson 



TO ERIC AND ORGANIZATIONS OPERATING 
UNDER AGREEMENTS WITH THE NATIONAL IN- 
STITUTE OF EDUCATION FUpTMER REPflO- 
DUCTlON OUTSIDE THE ERIC SYSTEM RE- 
.QUIR ES. PERMISSION Of TM E CQP^'^'fi*^'^ 
OWNER" 



Submitted in Partial Fulfillment 
of the Requirements for the MLS Degree 
Graduate School of Library and 
Information Science-, UGLA 



U S DEPARTMENT OF HEALTH. 
EDUCATION & WELFARE 
NATIONALINSTITUTEOF 
EDUCATION 



THIS DOCUMENT HAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIGIN- 
ATING IT POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 




ABSTRACT 



This paper provides data on and discusses the prepara- 
tion and delivery of eight computer -produced bibliographies 
for the use of entire classes of undergraduate students. The 
following are described and discussed: (1) the procedures 
followed in the preparation and delivery of each bibliography; 
(2) the benefits which are attributed to the use of the 
computer-produced bibliographies; (3) the costs in terms of 
dollars; (4) the costs in librarians' and clerical time; C5) 
the problems which have been or may be encountered in the 
operation of this service; (6) any suggested improvements in 
procedures; (7) additional uses; and (8) answers to questions 
regarding the continuation of this service. 

The data collection instrument consisted of a student 
questionnaire, a faculty interview, a reference librarian 
interview, and a library manager interview. Cost and time 
data for all phases of the search process were collected on 
the "operational procedures recording form". 

There were many variations in the preparation and in 
the use of the bibliographies. The cost and time figures for 
the seven searches varied greatly. 

• \ 

- THree types of benefits were described! XlT~Tong 
term benefits for students; (2) short term benefits for 
students; and (3) improvement in the quality of instruction. 
The main long term benefit is that the service provides the 
student with a positive library experience. Short term bene- 
fits include: (1) time saved in locating references allows 
more time for students to concentrate on reading and analyzing 
pertinent literature; and (2) a bibliography gives the students 
some place to begin when searching a new topic. The service 
also makes it possible for the professor to more easily update 
lectures with extensions of major concepts, and it may pro- 
vide instructors with a more effective method of meeting 
course objectives. 
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PURPOSE 



The purpose of this paper is to present and discuso 
the data collected on the '*pilot project to provide classes 
with computer-produced bibliographies" which is described in 
"Background of the Pilot Project to Provide Classes with 
Computer-Produced Bibliographies". The paper is descriptive 
and exploratory in nature- There has been no attempt to 
produce statistically rigorous data. 

The following will be described and discussed: 

(1) the procedures followed in the preparation and 
delivery of each bibliography; 

(2) the benefits which are attributed to the use of 
tfi e~ "c ompu^ by liTrt aiiT f acu^ 

bers and their classes during the Spring Quarter, 1976; 

(3) the costs in terms of dollars; 

(4) the costs in librarians' and clerical time; 

(5) the problems which have been or^^may be encountered 
in the operation of this service; 

(6) some suggested improvements in procedures; 

(7) additional uses; and 

(8) ansvers to questions regarding the continuation of 
this service. 



BACKGROUND OF THE PILOT PROJECT TO PROVIDE CLASSES 
WITH COMPUTER PRODUCED BIBLIOGRAPHIES 



In recent years, many academic libraries have started 
to offer computer-producfed j:bibliographic searches to their 
clients. Reluctantly, most of these libraries have had to 
institute charges for these services. While.. prof eissors and 
researchers will probably be able to afford these services, 
it is unlikely that individual students will ha've the neces- 
sary dollavs to take advantage of these costly services. 
While working as a student representative to the UCLA Academic 
Senate Library Committee, Mr. Jeffrey Griffith proposed "a 
rationale and a plan for using online bibliographic retrieval 
services to provide bibliographies tailored to the information 
needs of entire classes of students."^ This idea grew out of 
a b r oadef~c once^ 

library had to augment the instructional program of the campus 
Some subsidized funding for student use of automated biblio- 
graphic services was being contemplated. It was proposed that 
it would make good economic sense to prepare these biblio- 
graphic searches for the use of an entire class. In other 
words, on a cost per student basis the service would be more 
cost effective than if computer searches were made available 
on an individual basis only. As a result of Mr. Griffith's 
proposal, which was endorsed by the Academic Senate Library 



-"•"On Line Bibliographic Retrieval: An Instructional 
Resource for Classes, ED 121 242 (University of California, 
Los Angeles, March 22, 1976) , p. 1. 

10 



Committee, the University Library received funding from UCLA 
Vice-chancellor John C. Ries' "Innovative Teaching Fund** to 
carry out stage one of a proposed two stage pi?ot project. 

• The funding for stage one of the pilot projeci totaled 
$2500 and was to be divided among ten undergraduate classes 
$250 per class. The funding was to cover all costs incurred 
in the preparation of the class bibliography with the excep- 
tion of professional staff time. 

The original concept went far beyond simply producing 
a bibliography for student use. It called for the following 
steps to be carried out in the preparation of a class biblio- 
graphy. 

(1) After consultation with the instructor and possibly 
with the students, the reference librarian would formulate a 

s earch' strategy and: do- ar search~onr" one~ or~more"bibliographic — - 
data bases. 

(2) On receiving the offline printout the librarian 
would eliminate the obviously non-relevant citations. 

(3) The professor would review the output and check 
off references judged suitable for inclusion in the final 
bibliography. 

(4) The library would add call numbers and locations 
to the selected references, which in some cases included books 
and technical reports in addition to journal articles. 

(5) The citations would be cut and pasted so that the 
final product would include only the citations selected by the 
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instructor. Photocopies o£ the bibliography were to be made 
for each student. 

Specific procedures to be followed in preparation of 

the bibliography were to be determined by the reference librar- 

I 

ian in consultation with the instructor. 

By Mairch 31, 1976, the first day of the Spring Quarter, 
five library managers, whose libraries were, or would in the 
near future, be offering computer-produced bibliographic . 
searches, had committed themselves to recruit two professors 
each who were willing to participate in this project. The 
five libraries were t|ie Biomedical Library, the Education and 
Psychology Library, the University Research Library Reference 
Department, the Public Affairs Service, and the Physical 
Sciences and Technology Library. It was left up to each 
i ibr aiy t r eciiii t"pr Tt 
was requested that each professor be told that he/she would be 
asked to participate in an evaluation" of the pilot project at 
the end of the Spring Quarter, 1976. See Table 1 for a list 
of the participating libraries, classes, the data bases 
searched; and the subject of each search. 

Unfortimately, the Physical Science and Technology 
Library was unable to participate in the project. The Public 
Affairs Service participated in only one search during the 
Spring Quarter, 1976. One of the searches which the Education 
and Psychology Library participated in was used for two classes, 
(class numbers four and five oil Table 1). 
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Table 1 



Participating Ubrarles, Classes, Subject 
of Search, and Data Bases (s) Searched 



Class Subject of Search 



Mbrairy 



Education and Psychology 
Library 

University Research 
Library Reference Dept. 

Public Affairs Service 



Education and Psychology 
Library 



Education and Psychology 
Library 



Biomedical Library 



Biomedical Library 



University Research 
Library Reference Dept. 



Physiological 
Psychology Lab 

Presidential Nom- 
ination Politics 

Advanced Demo' ■ 
graphy 



Problems In In- 
structional 
Research , 

Cognition and 
Creativity In 
Education 

Plant Population 
Ecology 

Seminar In 
Genetics 

Political Theory 
and the Psychologists 



*'Rats and self<-st1mulat1on", 
"Rats and specific drugs" 

"Role of organized labor in 
national politics and In 
state primaries" 

"Population growth policy" 



"Cognition and creativity 
In education" 



"Pollination of flowering 
plants" 

"Ethical Issues In genetics" 



Data Base Searched 

Psychological Abstracts 

New York Times Informa- 
tion Bank 

Congressional Informa- 
tion Service, American 
Statistics Index 

Psychological Abstracts 

Psychological Abstracts 

BA Previews 
Medline 

Psychological Abstracts </i 



"Psychological conflict and 
ethical standards" 



"Cognition and creativity 
In education" 



DESIGN OF THE EVALUATION INSTRUMENT 



'The evaluation instriment is to consist of: 
(1) a student questionnaire; 
C2) a faculty interview; 

(3) a reference librarian interview; 

(4) a library manager interview; 

C5) an operational procedures recording form. 
Since the student is to be the end user of the biblio- 
graphy it is important to have feedback on: 

CI) benefits derived from the use of the bibliography 
included: (a) use of the bibliography; (b) savings 
in time; (c) usefulness for their purposes; (d) 
value of the librarian's discussion (,i£ applicable); 
and Ce) changes in library -habits resulting from 
participation in this project; 

C2) price students would consider reasonable if they 
were asked to share the costs of the bibliography; 

(3) continued use of an updated version of the biblio- 
graphy in the course; 

C4) means of access to the bibliography; 

C5) use of other reference tools; 

C6) manner in which bibliography was used; 

C7) any improvements students might suggest; 

C8) suggestions for other classes that might benefit 
from the use of a computer -produced bibliography. 

Feedback from professors will include: 

(1) judgments of the value of the bibliography for 
students and for faculty, and the factors to which 
this value or lack of value is attributable; 

(2) continuation of this, service; 

(3) suggested funding mechanisms; 
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(4) manner in which bibliography was utilized in 
teaching; 

(5) types o£ special processing given the biblio- 
graphies-; 

(6) any improvements they might suggest; 

(7) necessary lead time; 

(8) other ways computer-produced bibliographies might 
be used for classes; 

(9) for what kinds of classes would computer -produced 
bibliographies be most useful. 

Feedback from the reference librarians Cli^rarians who 
performed the, online search will include): 

(1) ways in which they think this service can help 
students; 

(2) role of this service in relation to the instruction 
of undergraduates in library use; 

(3) possible cutbacks in ongoing reference activity to 
allow more time to conduct searches for classes; 

(4) role of this service in relation to their role as 
a reference librarian; 

(5) procedures followed in preparation and delivery of 
the bibliographies including: (a) criteria for 
selection of the class; (b) did the reference 
librarian meet with the class; (c) topic of dis- 
cussion during the meeting, if applicable; (d) any 
problems students might have had; (e) any other 
means of communication with the class; (f) individ- 
ual attention to students; 

(6) for what kinds of classes would computer-produced 
bibliographies be most useful; 

(7) approximate length of a search for an individual 
client; 

(8) in what ways a search for a group differs from a 
search for an individual client. 

• » 

Feedback from library managers will include: 
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(1) the most important aspect of this service; 

(2) the ranking o£ searches for cl?,^5ses in relation 
to other new services which Ihe library manager 
would like to undertake; 

(3) possible cutbacks in ongoing reference activity 
to allow more time to conduct searches for 
classes; 

(4] professors* reactions when they were invited to 
participate in this project; 

(5) any noticeable impact on other library services 
which could be attributed to this service; 

(6) problems encountered; 

(7) number of searches the library manager *s library 
could handle per quarter. 

The "operational procedures recording form'* was used 

to collect times spent by librarians and clerical staff on 

various aspects of the bibliography preparation and delivery. 

See Appendix 1. 

METHODOLOGY 

The questions to be asked on the student questionnaires 
and in the faculty, reference librarian, and library manager 
interviews were developed by this author in conjunction with 
Peter Watson, Data Services Coordinator for the UCLA Library, 
and with the reference librarians and the library managers who 
participated. in this project. Griffith had suggested a list of 
questions to be included on the student questionnaire and in 
the professor interviews- -some of these questions were used in 
either the original or a modified form* The questionnaire and 
interview questions which finally evolved are shown in 

16 



Appendices 2, 3, 4, and S. The "operational procedures 
recording form" was suggested by Griffith and modified by the 
author . 

The author requested each participating reference 
librarian to ask the professor to have the students fill out 
the questionnaires during class time. Presumably if this pro- 
cedure had been followed the return rate for the questionnaire 
would have approached 100 percent. Unfortunately, in many 
cases, the questionnaire was only distributed during this time. 

The author set up appointments for the evaluation inter 
view with each professor, reference librarian, and library 
manager involved in the pilot project. In order to insure as 
much consistency in data collection as possible, each individ- 
ual (professor, librarian, etc.) was asked the same set of 
questions. Although it was desirable to cover the same ground 
with each person, it was equally important to obtain pertinent 
comments which would not have been elicited by the interview 
questions. To this end an informal interview arrangement 
enabled the author to ask follow up questions when a point was 
not clear. Also, before the first question was asked, the 
interviewee was asked to give their general impression of the 
project. Further the interviewee was encouraged to make any 
comments he/she thought would be important to the interviewer. 

Fifteen of the sixteen interviews were recorded. This 
was necessary to permit the interviewer to concentrate on con- 
ducting the interview rather than on taking notes. Later the 
taped interviews were transcribed, and the analysis is based . 
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on the transcriptions of the interviews. Closely related 
topics and ideas were discussed at different points of the 
interview. 

See Table 2 for the total number of participants and 
the number of respondents in each of the four groups. Table 3 
shows the total number of students in each class and the total 
number of questionnaires returned by the class. 



Table 2 

Number of Participants and Number 
of Respondents in Each of the Four Groups 





Group 


Total Number 
of Persons 
in Group (N) 


Total Number 
of Respondents 
in Group fn") 


Response 
Rate 
(percentage) 


(1) 


Student 


100 


58 


58% 


(2) 


Faculty 


7 


6 


85.7* 


(3) 


Reference 
Librarians 


6 


6 


100% 


(4) 


Library 
Managers 


4 


4 


100% 



J8 
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Table 3 

Number of Questionnaires 
Returned by Class 





Class 1 


Number of 
Students Enrolled 
in Class 


Number of 
Questionnaires 
Returned 


(1) Physiological 
Psychology Lab 


12 


6 


(2) Presidential 

Nomination Politics 


12 


12 


✓ 

(3) Advanced Demography 


8 


6 


(4) Problems in Instruc- 
tional Research 


9 


5 


(5) Cognition and 
Creativity m 
Education 


20 


5 

• 


(6) Plant Population 
" Ecology 


7 


7 


C7) Seminar in Genetics 


20 


17 


C8) Political Theory and 
the Psychologists 


12 




TOTALS 


100 


58 



Classes are presented in the same order as in 
Table 1. 

Questionnaires for this class disappeared in transit. 
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Presentation of Results 

The results are presented in tabular form in the 
following eight sections: 

(1) procedures followed in preparation and delivery 
of each bibliography including use; 

(2) benefits attributed to the use of the biblio- 
graphies ; ' . 

(3) direct costs calculated in dollars; 

(4) costs in terms of librarians' and clerical time; 

(5) problems which have been or may be encountered in 
the operation of this service; 

(6) suggested improvements in procedures; 

(7) additional uses; 

(8) questions .oncerning the cont2Jiuation of the 

service. 

The questions analyzed in each of the eight sections 
are shown in Table 4. Following each of the eight result 
sections is a summary and then a discussion of the .results 
presented in the tables which make up the section. 

The following abbreviations are used to indicate to 
whom a question was directed: 

(1) "S" indicates a student question; 

(2) "F" indicates a faculty or professor question; 

(3) "RL" indicates a reference librarian question; 

(4) "LM" indicates a library manager question. 



Table 4 

Questionii as They Relate to the Topics for Presentation and Discussion 



RcuTly miriiica Hiiriiy OptritioM) 
M Questions Qutitlons Llhrirlin Ninsgars' Praceduret 
teoU (towiKHn 31 Ijwstloni Questlooi Recording 
— tally] lifiMDiyi EttfflU, 

(1) Procedures followed SIHow did you hive F1 Hom Mt the bibllo- miCrlUrlefor 
In preparation end d^ access to the bibllo- graphyused? selection of class, 

livery of each biblio- graphy? 

flraphy. FS Special processing? 112 Did searcher 

S3 HoM did you use the leetMlth the class? 

bibliography? 

ins yhat Mas dis- 
cussed It the leet- 
ing? 

IIL4 Did students have 
any probleas? 

RLS Other coaMnlcatlon 
Nith class? 

116 Individual atten- 
tion? 



SI » question nuabar 1 on the student euastiOMiaire 
F1 " question nuolier 1 on the faculty tntervieM 



Table < (cont'd) 

' SluJwl Hciiny Renrence Clliriry opentionai 
Topic Qiiestiofli ()u6ttion$ LIbrirlin Ksnigers* Procedures 
(tependtx I) (AppendU 3) Questions Queitlou ' Itecording 
(ii ppendix 4) (Appendix 5) Fom 

(2) Benefits attribute SS Did you loajte and F2 On i scale ofl to IIL8 In lAat Miyi do LN) ttiat do you con- 
ed to the use of the read aiv iteis on the 7 (vith 1 being not you think this ser- ilder ,to be the MSt 
bibliographies. bibliography? How useful and 7 being vice can help stu* iifiortant aspect of 

Hnyl - nstusefuDiboir dentsjfanif} thii service? 

wuld you rate the 

$6 Do you think that value of this biblio- 119 Hon do you see thli 

having the bibllo* graphy for students? service In relation to 

graphy saved you To what factors 1$ Iwitructlon of under* 

tiM? HoM Ruch tlie? this attributable? graduates in library 



S7 In gcfierati njis the F3 On the sake scale 
bibllograpliy. useful as above» hoir Muld 
for your purposes? you rate the h\c 
Connent. of this blblloyiphy 

for yourself Ir^ pre- 

511 If the coefK^ter paring for students? 
search m discussed To Nhat factors Is 
w'llhyourclassbyi this attributable? 
Ilbrariani m the 

discussion valuabl'^7 

HOM? 

512 Has yuur partici- 
pation In this project 
altered the way In 
which you use the 
library? How? 



use? 



Ttble < (cont*Q} 



Topic 



Student 
Quntiom 



QuestloM 



wnnm 

Libfiriu 
QmsttOM 



Hbriry 



opentloni] 
Prociduns 
Kicordini 



(3) Costs in tim of 
doilirs. 



Costs iro nportid 
In coIm < 
(cbinies incum(l)i 



(<) Costs In Um of 
Ubririiitt^ md ctiri- 

Cll tiN. 



"Tisli". "Co^ilitid 
spent' colim, 



(S) ProblMS Mlikli 
kiife been or My be 
eocoiMtertd in the 
operition of this ser- 
vice. 



S4 Did you use other 
reference tools? 
ttiich ones? 



no ilppropriite 
funding ledunlsM} 



XL? Um does e seerch 
for e group differ 
f roi e SHrch for u 
individui] client? 



UB Professors' reac- 
tions vhei invited to 
pirticipite? 

UK lipact on other 
Tibrenr semicetl 

LM Problcn eAoounte^ 
ed so fir? 

IHS HoM MiDf seircbM 
next (|uirter? 



Tibli 4 (cMit'd) 

SUdMt hculty mrtrtflce ClSriiy optrittoMi 
Topic Qwitioiu Qmstim Ubririin Ninigtri' ' Pmedurtt 
(toxnii't l\ Itewdlx 3) qmitloitt' qutstlow tecoHlnt 
(Hwwdlx «) {kmM% 5) Fow 

(6) SiijKicsted 1^)ravi- S13 Art tiMn iny F4 iit luggtstioiu 
Mits in procKkint. provcMnb thit you Huld jiou nke for 

nuld suggHt in tbli i^jrovins this bib* ' 
Mrvici It you n- liogriphiciirviciT 
ciived it? Platse 

ccMnt. F8 In order to int^ 

griti this into your 
teidiina iditn wuld 
tl)t bibiiDgnpIv 
hive to bi rtiib'' 

(7) AddltloMl uses. SIO Other diss that F6 OthcriMys for using HU Types of 

could use OMputer* cottlute^pn)duced clisset? 
produced biblio* bibliogripliies for ' 
griphies? classes? 

F7 Types of classes? 



(8) Questions 
ceming the conttn* 
nation of the S6^ 
vice. 



$) Should diss coih 
tlnue to use updated 
bibllojinplv? 

SB ttoyldyoupay and 
idiat is reasonable 
price? 



F9 SixMild service be 
continued? 



UU Cut back on any 
refercMe activity 
to allow ior« tiie 
for tkis service? 

aiOltelatiOAof 
this service to ref* 
erence librarian 
rele? 



LW Cut back on any 
reference activity 
to allow iore tiia 
for this service? 

LN6 Asswino addi^ 
tional fundtni*- 
m services and 
ratine of searches 
for classes? 



Tihle M 



Procedure) Follwed In PrtiMrttlon ind Miveiy of uch Blbllojnptiy Including Use (by dist) 



-'^oImIciI Presldentlil 
Ptychology ' NMlNtlen 



Advinced , 

jggsailiL 



m\m In 
Instructional 



tiognliion and 
Creativity In 
Education 



Population 
Ecology 



Sailnar 
Genetics 



Political 
Theory and the 
Psychologists 



RLI Criteria for Prtife»orhad Suitabili^of Suitability of SaitabOlty of Suitability of Sultablllty'of SulUbllity of Suitability of 
selection of class, pi^vioiisly used course content, course content, course content, course content, course content, coune content, course content. 



RL4 Did students 
have aiiy problefls? 

RIS Other c(Mh 
ninlcatlon Mith 
classl 

RIG Individual 
attention? 



SI Access to 
bibliography? 



n Hoi^ifas 

bibliography 

used? 



F5 Specia] 
processing. 



t^/m searches 



IIL2 Did searcher No 
leet Mith class? 



RL3Nhatwas dis- 
cussed at leeting? 



YeS| students No 
received Instruc- 
tions at library. 



No 



Yes, students 
received In* 
structlon at' 
library. 



Yes I students 
rKOived 1n- 
itnictlon at 



Yes, students re- 
ceived Instruc- 
tion at library 
and cTafiroSii. _ 



Oenonstratlon 
coiputer and 
library search 
tectjnjques. 



Library search ' Coa^wter search Coiputer search 
techniques. and printout* and search 
techniques. 



Hone 



None Serial title Needed additional 
fllt>ffYHtior|S, ^purees. 



Cover letter. 



Cover letter 
SBiple clta- 
tloi^. 



Cover letter 
saa|i1e cita- 

tl0|)* 



Notes suggesting 
other sources. 



SOM 

students with 

additional 

sources. 



Soie students Soie students Helped soie stu- 

caae In to caie In to dents nIUi addl* 

thank refer- thank refe^ tional sources. 

ence Hbrarlan> enca librarian. 



Unserve and 
P^fessors 



Individual 
search. 



Individual 
copy. 



Reserve and 
professors 
J2HL_ 



Keserve and 
professors 
JSHL 



Individual co|v. Reserve. Individual search 



access 
toilurature 
p»v and 
po$t-e)<perl- 
aient. 



References for 
paper. 



Oral report 
on whether 
bibllograpiv 
Muld have 
been useful* 



(1) choosing 
topic for paper 

(2) references 

(3) update pro- 
fessor on as- 
pect of broad 



(1) references (1) references (1) choosing References, 
for oral roport for oral report topic for 

(2) professor (2) references (laper (2) ref- 
used to pr^ forrevlwof erences. 

pare lectures, literature. 



,/Dtbor index. 



HIcrofora 
locations 
added. 



01 Professor 
excluded non- 
relevant clta- 
tlons« 
(2) cut and 

(ii callniKH 
bers add»|. 



(1) Professor 
Indicated 
articles of cen- 
tral and perlpb 
eral Interest 
non-relevant 
teis excluded, 
cut and paste 
an * indi- 
cated that Iteis 
were owned by IICU, 



Call numbers 
added. 



*Class received bibliography after papers were coipleted. 
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Table SB ^ 

Procedures Followed in Preparation 
and Delivery of Each Bibliography Including Use 




Total 

Question Respondents Giving . Response 

; Response- 



S3 How did you 
use the biblio- 
graphy? 



51' 



42 

13 

8 
2 
1 



References for t^. 
paper. 

References for oral 

presentation. 

Ideas. 

Didn't use. 

Misunderstood the 

question. 



Some students used the bibliography in more than 
one way. 

Although 58 questionnaires were returned only 51 
students gave responses to this question. 
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SUMMARY: PROCEDURES FOLLOWED IN THE PREPARATION 
AND DELIVERY OF THE BIBLIOGRAPHIES INCLUDING THE 
USE TO WHICH EACH BIBLIOGRAPHY WAS PUT 

RLl^: Most (7 out o£ 8) o£ the classes were selected 
because the course content coincided with coverage o£ one or 
more data bases. One class was chosen because the professor 
fiiad utilized computer-produced bibliographies for his classes 
before. 

RL2: In 4 out of the 8 classes, the reference librar- 
ian met with the class at the library or in the classroom. 

RL3: One or all of the following took place at the 
meeting between the reference librarian and the class: (1) 
demonstration of an online computer search; (2) discussion of 
the computer -produced bibliography and the search strategy used 
in producing it; and/or (3) discussion of library search tech- 
niques. 

RL4: During the discussions with the students 2 of the 
4 reference librarians noticed that students needed help with 
understanding the serial title abbreviations in the biblio- 
graphy or with identifying additional reference sources. 

RLS: Instead of, or in addition to, meeting with the 
class, some reference librarians wrote explanations of CI) the 
way the bibliography was produced, (2) the organization of the 
bibliography, and/or (3) the citations contained in the biblio- 
graphy. The explanations were attached to the front cf the 
bibliographies. 

^Refers to questions on the interviews or questionnaires 
e.g., RLl " reference librarian questionnaire, question number 1 
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RL6: In at least two o£ the classes, individual stu- 
dents came to the reference librarian and asked for help 
locating additional types of reference materials, 

SI: Students had access to the bibliographies in one 
or a combination of the following ways: CI) on reserve in a 
library; (2) professor's copy; C3) individual copy of a class 
bibliography; and/or (4) for those who actually had a search 
tailored specifically to their needs- -a copy of their individ- 
ual search. 

Fl, S3: The bibliographies were used in a wide variety 
of ways including: CI) to provide references for a paper, oral 
report, or review of the literature; C2) to help students choose 
a topic for a paper; (3) to provide access to the literature 
prior to and after a class experiment; (4) to update professor 
on specific aspect of a broad topic; and (5) to help professor 
prepare lectures. 

F5: Examples of the types of special processing which 
some of the bibliographies received include: (1) compilation 
of an author index; (2) addition of call numbers of journals 
and/or books at UCLA; (3) addition of microform locations; and 
(4) remov?-l of non-relevant citations and inclusion of only 
relevant citations in the bibliography. 
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DISCUSSION: PROCEDURES FOLLOWED IN THE PREPARATION 
AND DELIVERY OF THE BIBLIOGRAPHIES INCLUDING 
THE USE TO WHICH EACH BIBLIOGRAPHY WAS PUT 

It is clear that there are many variations in the way 
in which these bibliographies were process and delivered. It 
is equally clear that no determination o£ the most effective 
(greatest benefits for least costs) manner of processing and 
delivery has been attempted. Further, there has been no 
attempt to differentiate between the benefits derived from the 
bibliography itself and those derived from the delivery (dis- 
cussion of bibliography and/or additional sources by a reference 
librarian) of the bibliography. 

For future study and in order to determine to which 
aspects of the service the indicated benefits can be attri- 
buted^ controlled experiments such as the following could be- 
set up. To determine the importance of discussion by the 
reference librarian, a class could be divided into thirds. 
One- third of the students would receive the bibliography and 
would attend a discussion by a reference librarian; one- third 
of the studnets would receive the bibliography and would not 
attend a discussion by a reference librarian; and one-third 
would not receive a bibliography but would attend a discussion 
by a reference librarian. All students would have the same 
assignment. The completed assignments would be graded by the 
professor without his knowing to which group each student 
belonged. 
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Table 6A 

Benefits Attributed to the Use of the Bibliographies (Students) 
Her ot 

Questions Respondents Yes I No \ 7 SD Range 

(R's') R's' ■ ■ ^"K's 



S5 (a) Did you 

locate and read I'to 
any items in the 57 48 U,l\ 9 ISM 8.10 7.50 38 

bibliography? items items 

(b) How many? (7) 



S6 (a) Do you 

think that having , 1 to 

the bibliography , 50 45 90,01 5 .lO.Oi 6ar 4.94 20 
saved you time? ^ hours hours 

(b) How much time? 

(Y) 



For the purposes of calculating 7, SD, and range for question S6 n = 18. 
The reason for this is that 60) oi the "yes" responses were not statistically 
usable, i.e., 10 students didn't specify time, 17 students gave non-numerical 
responses like many hours, several hours, days, etc, 



ts) 



Er|c 3ogl; 
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Table 6B 

Benefits Attributed to the 
Use of the Bibliographies (Students) 





Questions 


Number of 
Respondents 


Yes 


% 


No 


% 




CR's) 


R's 




R's 




S7 (a) In general, 
was the bibliography 
useful for your pur- 
poses? 2^ 
(b) Please comment. 


53 


48 


90.6% 


5 


9.4% 


Sll If the computer 
search was discussed 
with your class by a 
librarian, was the 
discussion valuable? 


29 


29 


100.01 


0 


0.0% 


S12 (a) Has your parti- 
cipation in this pro- 
ject altered the way in 
which you use the 
library? 
(b) How? 


56 


21 


37.5% 


35 


62.5% 



See Table 6C for narrative comments. 
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Table 6C 

Benefits Attributed to the Use of the Bibliographies (Students) 



Question 



Number of 
Responses 



Rfispreserttfltlvfe C^nt 



:6iiiiients received under yes « 
The bibliography saved time, 



S7 (a) In general, was 
the bibliography useful 
for your purposes? (b) 
Please coiient. 



S12 (a) has your partic 
Ipatlon In this project 



altered the way in Mh 
you use the library? 
How? 



ch 
b) 



11 
8 
5 
3 
2 

2 

22 



9 
5 
4 
3 



35 



3 

(5) 
7 



The bibliograpliy Mas generally helpful or useful. 
They would have missed references without the bibliography. 
The bibliography helped put their topic in perspective. 
Th^ were able to do other research because they had the biblio- 
graphy. 

It was convenient to have all the references in one place. 
No coimient. 



CoBiiients received under no . 

(1) The bibliography did not contain citations relevant to my topic of 

1|Dt6r6Sti 

2j No, but i think it will be. 
3) No comment. 

Comments received under ves. 



Increased awareness of resources . 
Increased efficiency In library use. 
Change In information seeking li; bits. 
No comment, 



Comments received under no . 
(1) No comment. 



^Comments received in response to questions 7 and 12, Actual conents not given. Comments have been 
classified Into categories. 



N) 



Tiblt 7 



Benefits ittrlbuted to the (lit of the Bibllogriphlei (Faculty) 



T SD lUngi ililKr, 



fictors RitlngiwreMtrHmtedtO! 



n (i) on I icili of 1 to 7 
(Nith 1 belni not uuful ind 
Ihiliig Hit itttfyl). hoM 
Hoold you riti thi viluo of 
thli blbllogripliy for ttudents? 
(T) (b) to «hit ficton Is 
this ittrlbuUbli? 



F3 (i) on I scilt of 1 to 7 
vith 1 being not useful ind 
7 being nit useful)) hov 
wuld you rite the vtlue of 
theilbliogripliy for yourself 
In prepiring for students? (i) 
(b) to Nhit fictors Is this 
ittrlbutible? 



NuriMr of ficulty HMbers giving response. 



(,S ,71 S.S ( (1) The tiM sived In kiting references Hde It possible 
to for student! to concentrete on rHdlng pertinent lit- 

7,D eriture or doing ectuil nseircb. 

2 (2) Provided students idthout Ideii for piper topics son 
pleci to focus. 

1 (3) The bibllogriplqf contilned references Nhlcb Mould 
hive been virtuilly l^wsilhli to find through stu* 
dents' none] search Procedures. 

1 (4) The cofrehensive bibllogriphy helped In defining 

Stentli) ireis for m reseirch. 
Is Is I better my to get ttudents Into the Titer- 
ituri thin other Mthods, 

5.67 2.21 1.0 4 (1) It ley help ichleve course objectives nrteffKtlve- 
to 1y thin previous lethods. 

7.0 3 (2) Eniblei the profeiior to lugmt lectures end ex- 
tend dJor concept! tilth lore recent end lore rele- 
, vent Infomtlon. leciuii of the n«ber of iin hours 
required, luy professors find It lifiosilbli to keep 
upNiththellUritureIn iivirilinai. 
Collects ill pertinent references In ihk plice. 
Bkiusi the professor could mn eisll/ oring hliself 
up to dite on I topic, he could five ion effective 
guldince to eich student In developing i reseirch pro- 
poiil. 



lii 
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Tablo 8 



Benefits Attributed to the 
Use of the Bibliographies (Reference tibrarians) 



Questions 



Number^ 



Responses 



RL8 In what ways do you 
think this service can 
help students, if any? 



RL9 How do you see this 
service in relation to 
instruction of under- 
graduates in library 
use? 



4 CD Students will be able to 
spend aore tiae actually 
using the material. 

2 (2) Sttideats have a Buch less 
frustrating eixounter with 
the library when there is 
a bibliogranhy with call 
numbers available. 

2 (3) If the project includes a 
visit with the libTarian, 
it would be a great help to 
students in learning how to 
use the library. 

1 (4) Because the librarian is 
involved with , the class 
he/she should be able to 
anticipate problems which 
the students might have. 

1 (5) A bibliography can provide 
students with background 
on a broad subject. 

1 (6) It can help students focus 

on a specific topic within 
a broader field* 

2 (1) The bibliography coul<! 

serve as the first step in 
introducing a student to 
the library. 

X (2) A. positive step to get out 
and show the student ehat 
the library wants to help 
them* -quite a different 
approach from making them 
come through the door and 
approach the reference desk. 

1 (3) This service may facilitate 
the facultys' encouragement 
of students to use the 
library. 

1 (4) The project provided the 
librarian with an oppor* 
tunity to try and help stu* 
dents who don't always have 
the background they need to 
use the library. 

1 (5) The project is primarily con- 
cerned with helping the stu* 
dent with an assignment for 
a class* It is secondarily 
concerned with teaching the 
student hew to use the 
library. 



Number of reference librarians giving responses 



34 



Table 9 



Benefits Attributed to the Use 
of tlie Bibliographies (Library Managers) 



Question 



Nxaittber^ 



Responses 



LMl What do you con- 
sider to be the most 
important aspect of 
this service? 



(1) This was an excellent way 
to introduce students and 
faculty to the concept of 
computer searching. 

(2) Each student benefited 
tremendously because they 
all got a library tour 
tailored toward what they 
were doing in the class. 
The discussion with the 
librarian covered refer- 
ence tools > services, and 
search techniques, 

(3) This project has brought 
to library-faculty and 
library- student relation- 
ships a new positive 
dimens ion- - incr eas ed 
library involvement with 
the instructional program. 

(4) The public relations value 
was great. 



Number of library managers giving responses 
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SUMMARY: BENEFITS ATTRIBUTED TO THE 
USE OF THE BIBLIOGRAPHIES (STUDENTS) 

SS: 84,2 percent of the student respondents located 
and read an average,. of 8.1 items from the bibliography. The 
range was from 1 to 38 items read. 

S6: 90.0 percent of the student respondents said that 
the bibliography saved them an average of 6,11 hours, Esti* 
mates of time saved ran from 1 to 20 hours t 

87: 90.6 percent of the student respondents said that 
the bibliography was useful for their purposes. The most 
often cited comment was that the bibliography had saved time 
(nine students) . 

511: 100 percent of the students who attended a dis* 
cussion by a librarian- said that the discussion was valuable. 

SIZ: 37.5 percent of the student respondents said that 
their participation in this project has altered the way in 
which they use the library. The most common change in library 
habits given was an increased awareness of resources (nine 
students) . 

^ SUMMARY: BENEFITS ATTRIBUTED TO THE USE 
OF THE BIBLIOGRAPHIES (FACULTY) 

F2: Professors rated the value of the bibliography for 
students at an average of 6.5 on a scale with a low of 1 and a 
high of 7. The range of answers was from 5,5 to 7.0. All six 
respondents agreed that the bibliography was helpful to students. 
The most often (5 out of 6) cited benefit was that the time 

36 
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saved in locating references made it possible:' for students to 
concentrate on reading pertinent literature, performing actual 
research, or consulting further reference tools. 

F5: Professors rated the value of the bibliography for 
themselves in preparing for students at an average of 5,67 on 
a scale from 1 to ?• The range of answers was from 1 to ?• 
Five out of six respondents agreed that the bibliography was 
useful for professors. The most commonly cited benefit (4 
out of 5) was that the bibliography might help achieve 
course objectives more effectively than previous methods. The 
second most cited benefit was that the bibliography enables 
the professor to augment lectures and extend major concepts 
with more recent and more relevant information. Because of 
the number of man hours required, many professors find it 
impossible to keep up with the literature in several areas, 

SUMMARY: BENEFITS ATTRIBUTED TO THE USE 
OF THE BIBLIOGRAPHIES (REFERENCE LIBRARIANS) 

RL8: All six reference librarians agreed that the scr- 
vici could help students. The most common benefit seen was 
that the students would be able to spend more time actually 
using the material (four reference librarians) , 

RL9: All six reference librarians saw the service in a 
positive manner in terms of instruction of undergraduates in 
library use. The most often given answer was that the biblio- 
graphy could serve as the first stop in introducing a student 
to the library (two reference librarians) . 

37 
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SUMMARY: BENEFITS ATTRIBUTED TO THE USE 
OF THE BIBLIOGRAPHIES (LIBRARY MANAGERS) 

LMli All four library managers agreed that the most 
important aspect of this service was that it was an excellent 
way to introduce students and faculty to the concept of com- 
puter searching, 

DISCUSSION: BENEFITS ATTRIBUTED TO THE 
USE OF THE BIBLIOGRAPHIES 

When we talk about benefits we are really trying to 
determine if this service improves the quality of instruction 
or the education which stiadents receive. If we consider the 
service in these terms, we see that yes it has a great deal to 
contribute in the way of improving instruction. The ways in 
which this service can improve the quality of the student's 
educational experience can be divided into three categories : 
(1) long term benefits; (2) short term benefits;, and (3) the 
quality of education as it is delivered by the instructor. " 

Long term benefits. From the broadest standpoint, the 
main long term benefit is that the service provides the student 
with a positive experience with the library. The following 
factors contribute to this positive experience: 

(1) This service shows students that the library- wants 
to help them. It is not always apparent to the student that 
the purpose of the library is to help them with their infor- 
mation needs. This type of outreach approach should help to 
counteract some of the negative impressions many students have 
developed in regard to using the library. 
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(2) This service shows students that the service can 
help them. The service is primarily conQemed with helping 
the student with a specific assignment. It is secondarily 
concerned with teaching the student how to use the library. 

(3) This service: (a) enables the student to bypass a 
certain amount of bibliographic di-udgery; and (b) allows the 
librarian to work closely with the class, and to anticipate 
problems that the class may encounter. 

(4) If the reference librarian discusses the biblio- 
graphy and/or further library sources, the student will learn 
of other specific sources and, in general, develop an increased 
awareness of resources and an increased efficiency in library 
use. 

Short term benefits . The project is primarily con- 
cerned with helping the student with a specific assignment. 
It does this in the following ways. 

(1) It gives the students some place to begin. It can 
provide background on a broad sr.bject. It can help students 
to focus on a specific topic within a broad field. 

(2) The time saved in lorating references allows more 
time for students to concentrate! or»> reading and analyzing the 
pertinent literature or doing re.f rr or In consulting other 
reference tools. 

(3) The bibliography may contalu references which would 
have been difficult or inspossille to lo;..i';a through students' 
normal search proceduvrcs . 

39 
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The quality of education as it is delivered by the 
instructor . The service can help the professor improve his 
quality of instruction in the following ways. 

(1) It makes it easier and less time-consuming to 
update lectures with the latest information* 

(2) The professor is able to better advise the student 
concerning the relevant and current literature in a broad sub- 
ject area. 

(3) The service may provide instructors with a more 
effective method of meeting course .objectives, tor example, 
it may be a better way of getting :i5tudents to r€>ad current 
literature. 
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Table 10 



Task Cost by Searcti 
(in dollars) 





1 


2 


3 


4 


5 


6 


7 


Task 
Total 


Hean 
Cost 
Per Task 


SEARCH COSTS 
Online time 

Ul 1 f IIIC y\ IlllUUld 

Service fee 
TOTAL 


$ 36.73 
72.10 
10.00 
118.63 


(168.75 (147.71 
27,25 
W 10.00 
178.75 184.96 


( 33.50 
66,70 
10.00 

110.20 


( 49,40 
66.00 
10.00 

125.40 


( 35,00 
35.00 


(144,20 
154,20 


( 615.29 
232.05 
60.00 
907.34 


( 87.90 
58.01 
10.00 

129.62 


Photocopying 


67,20 




16,68 


15.75 


28.70 


28.40 




156.73 


31.35 


2 

Clerical Labor 


63.58 


4.08 


44.54 


1,35 


27.20 


17,34 


40,80 


198.89 


28.41 


TOTAL Production 
Costs (exclud- 
ing professional 
librarian costs) 




















249,61 


182.83 


246.18 


127.30 


181.30 


80.74 


195,00 


1262.96 


180.42 


Professional Labor^ 


42.77 


49.67 


78.04 


21.30 


57.94 


56.91 


52.65 


359.28 


51.33 


TOTAL Production 
Costs (Includ- 
ing professional 
librarian costs) 




















(292.38 


(232.50 (324.22 


(148.60 


(239.24 


(137.65 


(247,65 


(1622.24 


(231,79 



The bibliography resulting from search number 4 was used for both class 4 and class 5. 

Clerical labor was figured at(4.,08/hour (Librarian Assistant II rate) direct labor costs, w 

^Professional librarian labor was figured at (8.52/hour (Associate Librarian IVjdirect labor 
costs, 



SUMMARY: COSTS 
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The average cost for the production of the computer- 
produced bibliographies (including clerical costs and exclud- 
ing professional librarian costs) is $180.42. The costs varied 
from $127.30 to $249.61. If the librarian costs are included 
the average cost goes up to $231.79, and the range becomes 
, $137.65 to $324.22, 

Online costs ranged from $33.50 to $147.71 and averaged 

$87.90. 

Costs for offline printouts ranged from $27.25 to $72.10 
and averaged $58,01. 

Total search costs (excluding labor) ranged from $35.00 
to $134.96 and averaged $129.62. 

Clerical labor costs ranged from $1,35 to $63,58 and 
averaged $28.41. 

Professional labor costs ranged from $21.30 to $57.94 
and averaged $51,33. 

Photocopying costs ranged from $0.00 to $67,20 and 
averaged $31.35 (excluding the instances of no charges). 

DISCUSSION: COSTS 

It is clear that the production costs of a computer- 
produced bibliography can be easily kept within the $250 esti- 
mates. It is also clear that the total costs for the production 
of a computer-produced bibliography can vary greatly as can the 
component costs which figure in ^jje total costs. 
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The production of these bibliographies should be thought 
of as a long-term investment and as in any long-term investment 
the costs should be spread over the years. The bibliography 
can be used by future classes, In this light, it can be said 
that the cost (including professional librarian costs) per stu- 
dent to date is $16.22. And by this time next year, the cost 
per student will have been reduced by one-half. 

In considering the $16.22 cost per student it should 
also be kept in mind that many of the classes involved injthis 
pilot project were quite small and in this sense unrepresenta- 
tive of undergraduate classes. If larger classes were chosen 
the cost per student would decrease proportionately as the 
number of students increases. 
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Table llA 



Reference Librarian Time Per Task by Search 



• 


1 


2 


Task Tl 
3 


ine by Search 


6 


7 


Task 
Total 


Task, 
Hean' 


Initial contact with 










• 










professor 


10 


75 


20 


20 


10 


35 


20 


190 


2M4 


Pre-search Interview 


60 


SO 


60 


JU 


ou 


20 


903 


370 


52.86 


Search formulation 


65 


SO 


165 


30 


20 


100 


60 


490 


70.00 


Online search 


31 


135 


90 


30 


38 


71 


96 


491 


70.14 


Exam^niitlnn nf niitniit (Hhn 
LAaiiiiimi luii ui uuiuuk ^iiuiii 








20, 


15 






lUU 


?fi fi7 

LVtVl 


Post-search discussion 


















a. with professor 


45 


40 


125 


20 


80 


30 




340 


56.67 


1 III 4 /i 411 

b. with class (Including 




















bibliographic lecture) 










110. 


1 80^. 


15 


205 


68.33 


Prepare cover letter and 




















sample citation 


60 








40 


10 




110 


36.67 


Supervision of clerical 




















tasks 


10 




60 




35 


10 


60 


175 


35.00 ' 


mO\ TOTAL 


301 


350 


550 


150 


408 


401 


371 


2531 





Time In minutes < 



Two reference librarians spent 40 minutes each 
'The class participated In the pre-search Interview. 
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Table 11 B 
Clerical Time Per Task by Search 



T ask Time by Seari — 
M ,12 5 4 5 6 7 Task Task, 

Total Mean 



Deliver output to 


30* 


30* 


5* 




6S 


2167 


professor 














Separate, collate 


80* 


70* 


20* 150 


200 


520 


104.00 


and copy output 














Special processing 














(add call nibers, 


720 


60 525* 


220 




600 2125 


425.00 


prepare author In- 














dex, cut and paste) 














Place bibliography or 












22.50 


Items on reserve 


15 


30* 






45 


Type cover letter or 














sample citation 


90* 




25 


45 


. 160 


53.33 


SEARCH TOTAL 


935 


60 655 


20 400 


245 


600 2915 





Time In minutes. 



*T1mes reported with an asterisk were actually completiid by a reference librarian but are 
classified as clerical tasks. 
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SUMMARY: TIME TABLE 

Librarian Tasks 

The amount o£ time invested in the pilot project on 
"professional tasks" ranged from a low of 2.5 hours to a high 
of 9.16 hours with an average of 5.93 hours. The tasks included 
in these time estimates are not the same for each bibliography, 
i.e., not every librarian composed a cover letter to be a.ffixed 
to the bibliography. 

The amount of time invested in the initial contact with 
the professor ranged from 10 to 75 minutes with an average of 
27.14 minutes. 

The amount of time invested in the pre-search interview 
varied from 20 to 90 minutes with an average of 52.86 minutes. 

The amount of time invested in the search formulation 
varied from 20 to 165 minutes with an average of 70 minutes. 

The amount of time invested in the online search varied 
from 30 to 135 minutes with an average of 70.14 minutes. 

The amount o£ time invested in examination of the out- 
put by the librarian varied from 15 to 45 minutes with an 
average of 26.67 minutes. 

The- amount of time invested in post-search discussion 
of the bibliography with the professor by the librarian varied 
from 20 to 125 minutes with an average of 56.67 minutes. 

The amount of time invested by the librarian in post- 
search discussion with the class varied from 15 to 110 minutes 
with an average of 68.33 minutes. 
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The amount o£ time invested in the preparation o£ a 
cover letter and/or sample citation varied from 10 to 60 
minutes and averaged 36.67 minutes. 

The amount o£ time invested in the supervision of 
clerical tasks ranged from 10 to 60 minutes with an average 
of 29 minutes. 

Clerica:! Tasks 

The amount of time invested in clerical processing 
tasks. varied from .33 to 15.58 hours with an average of 6.94 
hours . 

The time invested in delivery of the output of the 
bibliography to the professor varied from 5 to 30 minutes - 
with an average of 21.67 minutes. 

The time invested in the separation, collation, and 
photocopying of the output varied from 20 to 200 minutes with 
an average of 104 minutes. 

The time invested in the "preparation of the biblio- 
graphy" (clerical tasks such as adding call numbers, cutting 
and pasting, etc.) varied from 60 to 720 minutes with an 
average of 471.25 minutes. 

The time invested in the placement of the bibliograph- 
ies on reserve varied from 15 to 30 minutes with an average of 
22.5 minutes. 

The time invested in the typing of the cover letter 
and/or the sample citation varied from 25 to 90 minutes with 
an average of 53.33 minutes* 
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DISCUSSION: TIME TABLE 



The wide range in total librarian time spent on the 
search (2.5 - 9.6 hours) can be partially attributed to the 
different components which went into the delivery of the ser- 
vice. Another crucial reason for the time variations is the 
experience of the reference librarian with computer searching. 
Most of the reference librarians would be considered beginning 
searchers and as they gain expertise and confidence, they will 
probably spend less time in the tasks involved in the actual 
search process. Although the wide range in the time spent on 
the various component parts of the search is due in part to 
the inexperience of some of the reference librarians, even 
experienced searchers will complete parts of the search process 
with varying times. A mc I vivid example of these variations 
can be seen in a comparison between searches 5 and 6 (classes 
6 and 7). Both these searches were completed by the same 
highly capable reference librarian. Many variables contribute 
to the difficulty of and consequently to the time involved in 
the completion of the search process. For example, a search 
may be quite simple- -but if the user has difficulty expressing 
his information needs, the librarian will spend more time try- 
ing to understand what is being asked for- -so the pre-search 
interview time will need to be lengthy. 

The wide variations in clerical time can be attributed 
to the different types of processing given to the bibliographies. 
The issue of who is to be responsible for the clerical 
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processing is discussed in the section entitled "Discussion: 
Additional Aspects of Feasibility", 

It is clear that the operation o£ this service on a 
broader basis will require a substantial commitment in terms 
of librarians' time. The time data for searches S and 6 pro- 
vide the best basis for estimating costs in terms of time, 
approximately six hours of reference librarian time and from 
4 to 7 hours of clerical time per bi1)liography . 

It is useful to consider several trade-offs in dis- 
cussing the amount of time involved in the production of a 
computer-produced bibliography for a class, 

(1) One professor (class 1) estimated that the com- 
puter-produced bibliography saved him 18 hours of work. If 
one compares this 18 hours of professor's time to the 20.6 
hours of library time (5,02 hours of librarian's time and 
15.58 hours of clerical time) which it took to produce this 
bibliography, the amount of time involved takes on a different 
light. 

(2) Some of the timt, which went into the production of 
the bibliography would have gone into helping individual stu- 
dents learn, how to use basic reference tools and/or in a bib- 
liographic lecture for the class. 

It should also be pointed out that once the biblio- 
graphy is completed it can be used over and over again for the 
same class. It is a simple and inexpensive matter to update 
the bibliography once a year. 

49 
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Table 12A 

Questions Concerning Possible Problems 



Question 



Number^ 



LM3 Professors' re- 
actions when invited 
to participate? 

FIO What sources of 
funds do you consider 
to be the most appro- 
priate? 

LM4 What problems have 
been encountered so 
far? 



RL7 How does a search 
for a group differ 
from a search for an 
individual? 

LM5 How many searches 
next quarter? 



4 Immediate interest and curios- 
ity, 

5 Departmental funds. 

3 Could be required and sold 
through bookstore. 

1 Academic support funds. 

4 Amount of time invested in 
processing tasks. 

3 Amount of time invested in 
solicitation of classes. 

2 Coordination of frenetic 
schedules of professors and 
librarians , 

1 Lack of expertise and confi- 
dence on the part of the inex- 
perienced searchers, 

4 Broader in scope, larger retri- 
eval • 

2 No difference. 



2 Two searches. 

1 Two to five searches, asouiriing 
that they are treated as nor- 
mal data services searches. 

1 Four searches 



Number of persons giving particular response, 
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Table 12B 

Questions Concerning Possible Problems 
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Question 



Total No 
R's R's 



Yes 
R's 



No . 

Us ing 



Reference 
Tool 



S4 Did you use 
other reference 
tools? Which 
ones? 



57 



6 10.5% 51 89.5% 



32 Card cata- 
log. 

29 Indexes/ 

Abstracts 

11 Newspaper 
- indexes . 

12 Other. 



Table 12C 

Questions Concerning Possible Problems 



Question Total No % Yes % 

R's R's R's 



LM2 Impact on 44 4 100% 0 0.0% 

other library 

services? 



51 
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SUMMARY: QUESTIONS CONCERNING 
POSSIBLE PROBLEMS 

LM3: All four library managers found that most profes- 
sors who were contacted were immediately interested and curious 

FIO: Five out of siic pr^f e^sors suggested that depart- 
mental funds would be an appropriate mechanism for funding. 
Three professors suggested that students could be required to 
purchase the bibliographies in lieu of or in addition to text- 
books. 

S4: 89.5 percent of student respondents used one or 
more other type(s) of reference tools in addition to the 
computer-produced bibliography, 

LM2: There has been no discernable impact on other 
library services, 

LM4: All four library managers cited the amoxinfof 
time invested in processing tasks C^^i^otating bibliographies 
with call numbers, cutting and pasting, etc) as a problem. 
Three library managers found that the amount of time invested 
in the solicitation of professors had been a burden. Two 
library managers brought up the problem of the coordination 
of professors* and librarians^ frenetic schedules. 

RL7: Four of six reference librarians said that the 
search strategies for the bibliographies for classes were much 
broader than those for individual clients. Two reference 
librarians found that there was no difference. 

LM5: Two library managers said their library could do 
two searches during the next quarter. 
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DISCUSSION: QUESTIONS CONCERNING 
POSSIBLE PROBLEMS 

IM3: It is clear that some professors will be inter- 
ested in participating in this project. It should be stressed 
that professors will need explanations and illustrations of 
what a computer search can and cannot do. 

FIO: The question of funding needs further investi- 
gation. In addition to thinking that departmental funding 
would be the most appropriate funding mechanism, some profes- 
sors indicated that they would be willing to support this ser- 
vice in a budget request to their department chairperson. 

S4: This question was asked with the idea that if 
most students had consulted reference tools in addition to the 
bibliography that the issue of spoon-feeding would be refuted. 
It is quite evident that students did not use the computer- 
produced bibliography exclusively. 

LM2: Although as yet there has been no discernible 
impact on other library services there are several areas which 
probably would be effected if the service was offered on a 
broader basis. Reserve desks, inter-library loan services, 
and circulation statistics could expect to experience some 
impact. The developmer *■ of measures of impact on other 
library services will be an important aspect in the determina- 
tion of the true costs of this service. 

LM4: It is interesting to note the probable reasons 
underlying the most often brought up problems of time invested 
in processing tasks and time spent contacting prospective 
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professors. Neither one o£ these tasks are traditionally con- 
sidered part o£ the librarian's role. Most o£ the processing 
was or could rhave been done at the LAI I (clerical) level and 
provisions for reimbursement had been set up so that the 
libraries would be "paid back" for LAI I time. Presumably the 
reference librarian involved would have spent some time in 
supervising the LAII who carried out the clerical tasks. 

We are still left with the question of why the time 
spent on clerical tasks became such an issue. Possibly 
librarians did not want the library to become permanently 
saddled with the special processing end of this operation. Or 
the librarians may have truly believed that the special process 
ing would: not increase the value of the bibliography. However, 
the qnestion of the value of the special processing is still 
unresolved. 

If it is decided to continue with the special process- 
ing, it will then need to be determined whose responsibility 
it should become. Should the library, the individual profes- 
sor, or a special department assume responsibility for special 
processing? 

Solicitation of classes for participation in this pro- 
ject involves marketing of the library- -as one librarian put 
it- -"selling ourselves". Libraries and librarians are tradi- 
tionally passive. They expect users to flock to the library 
and consequently little effort is expended to aggressively 
seek users or to "advertise" library services. Marketing is 
traditionally associated with cg4»ercial sales and the "profit 
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motive", and many librarians' attitudes reflect the acade- 
micians' disdain for such mtindane functions. In addition to 
the amount of time involved, the above reasons probably con- 
tributed to the importance attached to this problem. 

Some librarians felt that the marketing of this service 
should have been the responsibility of the library administra- 
tion, 

RL7: It is not surprising to find that some of the 
search strategies for classes were broader than search strate- 
gies for searches for individuals. This must be considered in 
light of the problem of organization with several of the bib- 
liographies with a broad scope. The bibliography may lose its 
usefulness as its breadth increases. The issues of optimal 
length and breadth of the scope of the bibliographies should 
be investigated and it should always be brought to a profes- 
sor's attention during the discussion of the search topic. 
For more information about these issues see '^Discussion: 
Suggested Improvements in the Procedures Followed in the Pre- 
paration and Delivery of the Bibliographies", 

LM5: In light of the amount of time invested in these 
searches it is not surprising that no library manager would 
volunteer to have his/her library handle more than five searches 
for classes in a quarter. In answering this question, some 
library managers qualified their answers by saying that they 
did not want to be involved in the marketing of the service or 
in the special processing tasks. They wanted to again assume 
the passive stance. However, it is the writer's opinion thut 
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this service must be actively promoted by the library. Few 
professors know what can be accomplished with computerized 
bibliographic data bases and they are not going to find out 
unless the library shows them. In other words, it is the 
library's responsibility to market the service to its poten- 
tial users. Whether the marketing of a library service 
should be the library administration's responsibility or the 
individual library's responsibility should be decided on the 
basis of who can do the most effective job. 



56 



49 



Tabit 13 

Suggested Improvenents In the Procedures Followed In 
the Preparation and Delivery of the BlbllograpSles 



Option ' 

Responding 



Response 



$13 Are there any 
Wrovenents that 
you would suggest 
In this service as 
you received It? 
Please cooMnt. 
(Total respondents* 
52) 
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F4 What suggestions 
would you make for 
inproving this bib- 
liographic service? 
^Total respondents. 



P8 In order to Inte- 
grate this Into your 
teaching how many 
nonths ahead would 
we have to start? 
(Total respondents, 
6) 



No iflvroveaients needed. 

inPROVENEKTS SUGGESTED UHICH COULD BE IIICQR- 
POWTED IHTO TWIS SERVICE 

Improve the organization of the biblio- 
graphy and/or include an explanation of 
the bibliography. 

the bibliography should be edited, 
handout with serial title abbreviations 
should accompany bibliography, 
further reference sources should b« 
indicated. 

make printout more readable by under- 
lining titles. 

allow students to choose own terms, 
allow students to see unedited search, 
reference librarian at desk did not know 
how to use printout. 



(1) 

!i! 

(4J 
(5) 
6 



INPROVEMENTS SUGGESTED IN GENERAL LIBRARY 

msmi 

1} Improve sue of use of microfilm. 

2) make availability of service known. 

3) Improve cireuliatlon procedures. 

SUGGESTED IMPROVEMENTS RELATED TO OATA BASE 

1) make service ;1ess expensive. 

2) Improve Indej^lng. 

3) Include more/recent articles. 

1) none. 

2) start Hrlltr. 

3) organlutloH of bibliography should be 
explained* and Improved. 

(4) general search at beginning of c1ass» 
then specific searches as students 
dKlde on topics for papers. 

(1) bibliography should be ready on first 
day of class. Begin approximately one 
month ahead. 

(2) bibliography should be ready M weeks 
before the quarter begins. 



57 



50 

SUMMARY: SUGGESTED IMPROVEMENTS IN THE PROCEDURES 
FOLLOWED IN THE PREPARATION AND DELIVERY 
OF THE BIBLIOGRAPHIES 

313: Twenty«six students suggested a number of 
improvements. The most often suggested improvement (eigh-t stu- -" 
dents) was that the organization of the bibliography should be 
improved and/or explained. The second most often suggested 
improvement was that the bibliography should be edited. 

F4; The most often suggested improvement by profes- 
sors was that production of the bibliography should get started 
earlier. One professor said that the organization of the bib- 
liography should be explained and/or improved. 

F8: Five of the six professors said that the biblio- 
graphy should be ready for distribution to students on the first 
day of class. One professor wanted to have the search 8 to 9 
weeks before the beginning of Cxass. 

DISCUSSION: SUGGESTED IMPROVEMENTS IN THE 
PROCEDURES FOLLOWED IN THE PREPARATION 
AND DELIVERY OF THE BIBLIOGRAPHIES 

It is suggested that the underlying reason why organi- 
zation of the bibliography came to. the forefront as a problem 
is that some of the bibliographies contained from 600-700 cita- 
tions. These 600-700 citations were given to the students just 
as the computer printed them out- -organized only by date of 
entry into the data base. 

There are two discernible issues here; CI) number of 
citations included in a bibliography; and (2) the organization 
of the citations -within the bibliography. 
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Several questions should be considered with regard to 
the number of citations included in a bibliography: 

(1) Would organization be a problem with fewer cita- 
tions? 

(2) What is the optimal number of citations? 

(3) What factors lead to a huge retrieval? 

(4) What methods can be employed for limiting retrieval? 

(5) What is the relationship between huge retrieval and 
offline printing costs? 

Organization would probably become less of a problem 
as the number of citations decreased. And, the task of impos- 
ing some form of organization would become easier and less 
time-consuming as citations decreased. The main purpose of 
indexes and abstracts is to make citations easier to get at 
by assigning them to subject categories. It could be argued 
that giving students bibliographies consisting of se^^eral 
hundred unorganized and unedited citations is making biblio- 
graphic retrieval even more difficult than it normally is. 
On the other hand, some instructors may think that it is an 
important part of a student *s education to learn to impose 
some order on chaos. 

The usefulness of the bibliography for students will 
probably vary with the number of citations included in the 
bibliography and with the organization of the citations. It 
can be hypothesized that as the number of citations in the 
bibliography increases so must the degree of organization 
increase. 
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There are several factors which could lead to retrieval 
of a large number of citations: 

(1) the topic as stated by the professor was very 
broad; 

(2) the amount of publication in a narrow field is 
prolific; 

(3) the reference librarian failed through inexperi- 
ence to anticipate the large retrieval; 

(4) the reference librarian told the professor that 
retrieval might be quite large but the professor 
insisted on a comprehensive retrieval. 

There are many ways in which retrieval can be limited: 

(1) by date of publication (i.e., 1975 and 1976 
articles only) ; 

(2) narrow search terms can be used; 

(3) certain aspects of a broad topic can be excluded; 
and 

(4) foreign publications can be excluded. 

The size of retrieval is the single most important 
factor in the cost jf the offline printout. As can be seen 
in Table 10 the offline printout costs can exceed the online 
computer time. There are various ways of charging for offline 
printouts ($0.25 per citation, $0.20 pcjr page) and in most 
cases, offline printout charges increase with the number of 
citations printed. 

The editing and the organization of the bibliography 
will be the key factors in the ease with which the 
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bibliography is used and in cases of very large retrieval, will' 
probably determine the usefulness of the bibliography. 

By editing I mean the selection of articles for inclu- 
sion in the bibliography. By organization is meant the way 
in which the citations are organized within the bibliography. 
Since the percentage of relevant citations retrieved in a com- 
puter search can vary from 0 to 100 percent, professors and 
students must be made aware of this fact if a bibliography is 
to remain in its unedited state. 

Who should be responsible for the editing and organi- 
zation of a bibliography? 

The reference librarian is responsible for making the 
organization which he/she has imposed on the bibliography 
clear. Toward this end, a tabr;; of contents should be pro- 
vided. ' 

In some cases a tabl'i of contents to the bibliography 
will be sufficient--especially if the search strategy consisted 
of several search statements or m'- lules each of v/hich retrieved 
a small number of citations. :. u the event that one search 
statement retrieved a large number cf citativus it may be 
necessary for the proifessor or teaching assisusrit to impose 
some degree of organization on the citat'Ions. 

One professor (class 6) edited and organized the bib- 
liography in the following way: (1) irrel-ivant citations were 
excladfid; and (2) citations were div.'ded into "articles of 
central interest" and "articles of p'': ripheral interest". In 
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order to give the class a feeling for what the articles in 
the bibliography contained- -the professor spent an hour of 
class time telling something ?5bout each article- -was it periph- 
eral or central to the topic of the class,.; was it hypothetical, 
speculative, or did it contain hard-core data or was it a 
major review article. This professor was concerned that his 
i students avoid spending slot of time looking at peripheral 
articles . 

The amount of edir.mg and organizing a bibliography 
receives should depend on ths sophistication of the students, 
course objectives, length of oibliography, and the importance 
which the professor places on w'le inclusion of only relevant 
citations and the manner in which they are orgemized. 
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Table 14A 
Uses 



Question Number - Response 
• Respondents 



F6 Other ways for 2 Students could do own search, 

using computer 
searches for 
classes? 



F7 Types of classes? 5 0) any class which requires a 

paper or oral report. 
1 (2) any class in which a consider- 

able amount of reading in 
original journals is required. 



RL12 Types of 2 (1) any class with exception of 

classes? those based on only textbooks 

and lectures. 
1 (2) small classes in which course 

content and data base are 

matched. 

1 (3) graduate and u;yper-di vision 

small classes. 

1 (4) classes with tightly defined 

focus. 

1 (5) seminars in which current infor 

mation is important. 

1 (6) large undergraduate classes (a 

large. bibliography could have 
several modules so students 
would have a broad choice of 
topic). 
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Tal?le. 14B 
Uses 



Question 


R's 


Yes 


% 


No 


% 


Note 






R's 




R's 






SIO Other classes 
that could use 
computer-produced 
bibliographies? 


51 


34 


66,7% 


17 


3Z.Z% 


22 different 
courses in 9 
subject areas 
listed by 29 
students . 



SUMMARY: USES 

F6: Professors could think of only one additional way 
in which searches for classes could be employed. Students could 
write their own search strategies and then run their own 
searches* 

F7: When asked what types of classes computer-produced 
bibliographies would be most useful for professors answered in 
the very broad terms of "any class in which a paper or oral 
report is required or which requires a large amount of reading 
in the journal literature"* 

RL12: When reference librarians were asked for what 
types of classes computer-produced bibliographies would be most 
useful, two reference librarians thought that they would be use- 
ful for almost any class with the exceytion of those based on 
only textbooks and lectures. Other responses given by reference 

A 

librarians were much narrower in terms of specific types of 
classes, i^m 
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SIO: 66.7 percent of student respondents could think 
of other classes that could use computer-produced biblio- 
graphies. Twenty-nine students liisted a total of 22 differ- 
ent courses in nine subject areas. 

DISCUSSION: USES 

The suggestion that students could write their own 
search strategies and run their own searches is excellent. 
There is no doubt that this would provide a highly valuable 
learning experience for students. However, it would probably 
require an even greater amount of the librarian's time. This 
possibility should be explored. 

In considering professors* and librarians* responses 
to "for what types of classes would computer-produced biblio- 
graphies be useful'* one notices that professors give the 
broadest possible answers while reference librarians tend to 
give answers in terms of specific types of classes. Each, 
reference librarian is probably answering the question. in 
relation to the problems he/she encountered in one or two 
specific searches. The professor is probably unaware of the 
problems which the librarian may have encduntered. The type 
of class is yet another variable to be considered in terms 
of the benefits and the costs of this service. 
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Table ISA 



Questions Concerning Continuation of Service 



Question 


Total 


Positive 


■ % 


Negative 


% 




R's 


R's 




R's 




F9 Should service 


6 


6 


m% 


0 


0.0% 



be continued? 

S9jShou1d class 57 52 91.2% 5 8.8% 

continue to use 
up<^ated biblio- 
graphy? 



Table 15B 



Questions Concerning Continuation of Service 



Question 


Total 


Positive % Negative % 




SO Range 




R's 


R's . R's 






S8 (a) would you 
pay? (b) what 
would be a 
reasonable 
price? 


45 


34 75.695 11 24.4% 


$4.14 


$2.59 $0.20 
to 
$10.00 



For students who would pay. 
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Table 15C 

Questions Concerning Continuation of Service 



Question 



Total Positive % Negative % 
R's R's R's 



RLll Cut back on 
any reference 
activity to 
allow more time. .? 

LM7 Cut back on 
any reference 
activity to 
allow more ; 
time . .? 



16.7% 



0.056 



83.3% 



4 100.0% 



Table 15D 

Questions Concerning Continuation of Service 



Question 


Total 
R's 


Number Response 


RLIO Relation of 






this service to 


6 


6 Logical extension of reference 


reference librarian 




service. 


role? 
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Table ISE 

Questions Concerning Continuation of Service 



Question Total 

R's 



LM6 Assuming additional 4 
funding— new services 
and rating of searches 
•for classes in rela- 
tion to the new 
services? 



Services Listed 



(1) more in-depth reference. 

(2) extend reference hours. 

(3) conduct one to four hour seminars 
for graduate students. 

(4) give orientations for under- 
graduates, 

(5) formalized program to orient 
Research and Te.'ch1ng Assistants. 

(6) deepen and enrich reference collec- 
tion. 

(7) 'Hork more closely with departments. 

(8) develop AV kits for reference tools. 



■ Data services in general and computer-produced bibliographies 
for classes specifically were rated in the lower one-half of 
the services listed above. 

SUMMARY: QUESTIONS CONCERNING CONTINUATION 

OF SERVICE 

F9: 100 percent of the professors agreed that the ser- 
vice should be continued. 

S9: 91.2 percent of student respondents t.hought that 
the class should continue to use an updated bibliography. 

58: 7.S.6 percent of student respondents said that they 
would ray an average of $4.14 for the bibliography. The range 
was from $0.20 to $10.00. 

RLIO: 100 percent of the reference librarians sav; this 
service as a locial extension of reference service. 
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RLll: 83.3 percent o£ the reference librarians said 
that they would not consider cutting bac^^on any ongoing refer- 
ence activity to allow more time for this new service. 

LM7: 100 percent of the libra*^ managers said that 
they would not consider cutting back on any ongoing reference 
activity to allow more time for this new service. 

LM6: When asked to list new services that they would 
want to undertake if they received additional fujiding, .itrary 
managers listed several services most of which were extern uons 
of already existing services. When asked to rank computer- 
produced bibliographies for classes in relation to the other 
services listed, they were rated in the lower half. 

DISCUSSION: QUESTIONS CONCERNING THE 
CONTINUATION OF SERVICE 

On the one hand we have an enthusiastic response to the 
service by its prospective users. On the other hand, we have 
what seems to be a negative response on the part of the groups 
who are the prospective providers of the service. Many reasons 
can be offered to explain these seemingly cool responses. 

(1) Librarians in some reference departments feel that 
their reference departments have reduced the services they^ 
offer to a bare minimum. For this reason, these same librarians 
feel that the services which they do offer are so basic that 
there can be no possibility of cutting back on them. And con- 
sidering this prevailing attitude, it is not surprising that 
when library managers were asked what new servir.es they would 
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like to undertake if new funds w^re allocated for reference 
activitifas that most of the services named were improvements 
or expansions of services already offered. This new service 
seems to be viewed as frosting on a cake which does not have 
any sugar in it. 

(2) The reference staff of each library is of a fixed 
size and each person has certain responsibilities within the 
reference department. Because of these fixed responsibilities 
it is difficult to decide who would have responsibility for 
the new service. 

(3) Any reconsideration of priorities is time'Consuming 
and can be painful. It is difficult to admit that there may 
be new and perhaps more effective ways of offering library 
service. This new service represents a complete departure from 
the traditionally passive role of the library in instruction 
within the university. Whenever change is contemplated there 
is a potential threat to the persons involved. 

(4) It is likely that many librarians find it difficult 
to see beyond the piles of work which participation in the 
development and implementation of this pilot project has created. 
None of the libraries involved received any compensation (in 
terms of professional staffing) for participation in this pilot 
project. The reference librarian had to neglect other responsi- 
bi'ities to make time for working on the prlot project. 

(5) Some librarians are not convinced that this service 
is worth pursuing. They may feel that the traditional methods 
of library service are more effective. 
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(6) There seemed to be a prevailing fear among librar- 
ians that if this pilot project succeeded, the library admin- 
istration would ask the libraries to offer this service on a 
much broader basis without adequate provision for additional 
staffing. It is not surprising that new demands would be 
looked upon as a hardship unless new funds accompany the new 
demands • 

(7) Some reference librarians had frustrating experi- 
ences for several reasons: (a) lack of experience and conse- 
quent lack of confidence in their ability to perform ade- 
quately; (b) guilt over neglect of normal responsibilities; 
(c) time pressures because the bibliographies had to be com- 
pleted quickly in order to be of use to students. 

(8) Many librarians felt that the library administra- 
tion should have assumed the responsibility for recruiting the 
participating classes. It was generally agreed that the 
library should not be put in the position of having to sell 
itself. And if selling was necessary it should not be the 
task of the individual libraries but rather an administrative 
responsibility. 

CONCLUSION 

It is clear that this service is highly beneficial for 
both students and faculty. The costs incurred in the produc- 
tion of the computer-produced bibliography can be held within 

> ■ f. 

$250.f'7The additional costs in terms of librarians' time is 
high. A sizable commitment in librarian time will be necessary 
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if this service is to be offered to a significant number of 
classes each quarter. Consequently, either a reexamination 
of library priorities and a reallocation of library staff 
responsibilities, or new staffing may be called for. 

As a result of this study many areas for further study 
have been identified* Some of the most important issues for 
further investigation are: 

(1) What is the relationship between the use of the 
bibliography, the discussion of the bibliography by a reference 
librarian, and the discussion of library reference tools by a 
reference librarian, in maximizing the benefits experienced by 
the students? Should all three be included in the delivery of 
the service? Could the service be provided in the same manner 
as a search for an individual client and still be effective? 

(2) Most of the classes used in the pilot project are 
not really representative of undergraduate classes, A more 
rigorously controlled study should be carried out and classes 
should be chosen to represent major types of undergraduate 
classes « 

(3) Measures o£ impact on other library services 
(reserves, inter-library loan service, circulation, etc) need 
to be developed. 

(4) What part does the special processing play in mak- 
ing the bibliographies effective as teaching tools? 

(5) Long-term benefits can rnly be guessed at. Attempts 
could be made to measure long-term i^^iiefits. 
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(6) It may be found that classes in certain subject 
areas can benefit more than others by using this type of bib- 
liography. Life sciences data bases are at a more advanced 
stage of development than social sciences data bases, 

C7) What is the optimal length for the computer- 
produced bibliography and to what degree must the bibliography 
be organized to be useful to unde'T'graduate students? 

In conclusion, this type of service gives faculty 
members an opportunity to malce use of the library in general 
and the reference librarians specifically as an instructional 
resource. Conversely, this type of service gives librarians 
a chance to become more directly involved in the instructional 
process. 
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Appendix 1 , 

•'!■ 

r 

Operational Procedures Recording Form 



Instructor 



Search Title 



Task 



Completed By 
(print name) 



Time Spent 
(nearest 5 
minutes) 



Charges Incurred 
(e,g,, search, 
printing, 
photocopying) 



Problems 
Encountered 



Conents 
(e.g., action 
taken, sug- 
gested solution) 



met 0^ 
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^ Appendix 2 

University of California Los Angeles 

University Library 
and 

Graduate School of Library and Information Science 
Dear Student: 

This has been an experimental project conducted by the Library 
to investigate the use of computer-produced bibliograp ' es for 
classes. To help us evaluate the program, please fill out the 
attached questionnaire and return it to your professor. If 
you would like to discuss any aspect of the project further, 
please call me at 392-5524 or leave a message at the Graduate 
School of Library and Information Science, Room 120, Powell 
Library, Campus. 

Thank you very much for your help. 



Carol R, Nelson 
Graduate Student 
GSLIS 



QUESTIONNAIRE FOR STUDENTS 

1. How was the bibliography made available for your use? 
( ) On reserve 
( ) Own copy 

( ) Shared copy 

( ) Other, please specify 



2. Was this adequate? 
( ) Yes 
( ) No 

If no, what was the primary reason for this?_ 



3. How did you use this bibliography? (E.g., to get references for term 
paper, to get ideas for a topic for your paper, etc.) 
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4. Did you consult other reference tools beyond this bibliography? 
{ ) Yes 

( ) No 

If yes, please check off reference tools consulted: 
( ) Card Catalog 

( ) Indexes or abstracts 
( ) Newspaper indexes 

( ) Other, please specif y 

5. Did you locate and read any items in this bibliography? 
( ) Yes 

( ) No 

If yes, approxiimately how many items did you locate and read? litems 

6. Do you think that having the bibliography saved you time? 
( ) Yes 

( ) No 

If yes, how much time do you think it saved? h ours 

7. In general, was the bibliography useful for your purposes? 

( ) Yes 

( ) No 
Please comment. 



8. If students were asked to purchase the bibliography rather than 
receiving it for free, what would you judge to be a reasonable price? 

9. Do you think this course should continue to use an updated version 
of this bibliography? 

( ) Yes 
( ) No. 

10. Are there other classes you have had that you think could have made 
use of computer searches? 

( ) Yes 
( ) No 

If yes, please list classes and/or instructors: 

11. Was the computer search discussed with you or your class by a 
librarian? 

( ) Yes 
( ) No 

If yes, please comment on whether this was valuable or not: 

12. Has your participation in this project alterui the way in which you 
use the library? 

( ) Yes ^ 

( ) No 
If yes, how? 
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13. Are there any improvements that you would suggest in this service as 
you received it? 
( ) Yes 

{ )'No 
Please comment: 



Thank you again. Your help is appreciated . 
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Appendix 3 
Interview Questions for Professors 



!• How did you use this bibliography in your teaching? 

2, On a scale from 1 to 7 (with 1 being not useful and 7 being 
most useful) , how would you rate the value of this biblio- 
graphy for students? To what factors is this attributable? 

3, On a scale from 1 to 7 (with 1 being not useful and 7 being 
most useful) , how would you rate the value of the biblio- 
graphy for yourself in preparing for classes? To what 
factors is this attributable? 

4, What suggestions would you make for improving this biblio- 
graphic service? 

5* What type of special processing did the bibliography 
receive? Was this special processing worth the effort? 

6. In what other ways could you see employing computer- 
produced bibliographies for classes? 

7. For what kinds of classes do you think computer-produced 
bibliographies would be useful? 

8. In order to integrate this into your teaching, how many 
months ahead would we have to start? 

9. Would you like to see the library continue this service? 

10* If so, when soft money is no longer available for this 
project what sources of funds do you consider to be the 
most appropriate? 
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' Appendix 4 
Interview Questions for Reference Librarians 



1. On what criteria did you base your selection of classes for 
participation in this project? 

2. Did the professor schedule time for you to come to the class 
or for the students to come to the library? IF YES, ASK 
QUESTIONS 3 and 4, IF NO, SKIP TO 5. 

3. What did you talk about when you met with the class? 

4. Did the students need help with anything in particular? 

5. Did you communicate with the class in any other way? 
(cover letter, etc) 

6. Did any students receive individual attention? 

7. How did the fact that the project search was for a group 
rather than an individual affect the search? 

8. In what ways do you think this service can help students, 
if any? 

9^ How do "ybU~'see~ this' service~in~relatxon~ to^~ra 
undergraduates in library use? 

10. How do you see this service in relation to your role as a 
reference librarian? 

11. If you were in a position to make a decision--would you 
consider cutting back on any ongoing reference activity 
to allow more time for this new service? 

12. What kinds of classes do you think computer-produced 
bibliographies would be useful for? 
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Appendix 5 

Interview Questions for Library Managers 



!• What do you consider to be the most important aspect of 
this service? 

2, Has there been any noticeable impact on other library 
services as a result of this service? 

3. How did professors react when invited to participate in 
this experimental project? 

4. What problems have been encountered so far? 

5, If funding for this project continued to cover the cost of 
the search and its processing but not librarian's time, 
approximately how many searches a quarter would your unit 
be able to handle? 

6, Assuming your library was to receive additional funds for 
reference services, what new services would you want to 
undertake? What priority would you assign this new service 
in relation to these other new services? 

7. Would you cons id^^ any ongoing reference 
services to allow more ti^e for this new service? 
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